Digital signal compression is widely used in many multimedia applications and devices. Digital signal compression using a coder/decoder (codec) allows streaming media, such as audio or video signals to be transmitted over the Internet or stored on compact discs. A number of different standards of digital video compression have emerged, including H.261, H.263; DV; MPEG-1, MPEG-2, MPEG-4, VC1; and AVC (H.264). These standards, as well as other video compression technologies, seek to efficiently represent a video frame picture by eliminating or reducing spatial and temporal redundancies within a given picture and/or among successive pictures. Through the use of such compression standards, video contents can be carried in highly compressed video bit streams, and thus efficiently stored in disks or transmitted over networks.
MPEG-4 AVC (Advanced Video Coding), also known as H.264, is a video compression standard that offers significantly greater compression than its predecessors. The H.264 standard is expected to offer up to twice the compression of the earlier MPEG-2 standard. The H.264 standard is also expected to offer improvements in perceptual quality. As a result, more and more video content is being delivered in the form of AVC(H.264)-coded streams. Two rival DVD formats, the HD-DVD format and the Blu-Ray Disc format support H.264/AVC High Profile decoding as a mandatory player feature. AVC(H.264) coding is described in detail in ISO/IEC 14496-10:2009, “Information technology—Coding of audio-visual objects—Part 10: Advanced Video Coding, Edition 5” May 13, 2009, which is incorporated herein by reference. A copy may be downloaded at the following URL: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=52974.
Many codecs make use of different types of coding of frames. Examples of different frame coding formats include Intra-coded frames (I-frames), predictive coded frames (P-frames) and bi-predictive coded frames (B-frames). In general terms, an I-frame is coded without reference to any other frame. An I-frame can be decoded independent of the decoding of any other frames. I-frames may be generated by an encoder to create a random access point that allows a decoder to start decoding properly at the location of the I-frame. I-frames generally require more bits to encode than P-frames or B-frames.
P-frames are coded with reference to one or more other frames, such as an I-frame or another P-frame. A P-frame contains changes in the image from one or more previous frames. Decoding a P-frame requires the previous decoding of one or more other frames. P-frames require fewer bits to encode than I-frames. B-frames are similar to P-frames but contain image differences with respect to both previous and subsequent frames. B-frames can be coded in some prediction modes that form a prediction of a motion region within the frame by averaging the predictions obtained using two different previously-decoded reference regions. B-frames require fewer bits to encode than I-frames or P-frames.
The coding of video streams into bitstreams that contain I-frames for transmission over the Internet is subject to certain problems. One problem is compression delay. Even though an I-frame typically requires more bits than a P-frame or B-frame it takes more time to compress and encode a video image as a P-frame or B-frame than as an I-frame. Another problem is referred to as bit-rate jitter. Because I-frames consume much more bit counts than P-frames or B-frames, the bit rate for producing encoded pictures is uneven. It would be more desirable to have a smoother bit rate.
To overcome the compression delay and bit-rate problems it would be desirable to eliminate the use of I-frames in video coding. 1. Intra Refresh (IR):
Intra Refresh (IR) technique was first introduced in early video codecs, such as MPEG-1 and MPEG-2. Intra Refresh is a well-known and well-developed encoding feature for providing error resilience in transmitting compressed video contents over error-prone network environments. The basic idea is to forcefully assign an INTRA mode to some macroblocks in inter-predicted pictures (e.g., P-frames) in a way that each macroblock in the picture will be encoded as an INTRA macroblock at least once to confine error propagation if errors occur. Another benefit of employing this technique is that it codes all video frames as P-frames, which alleviates bit rate fluctuations resulting from different types of coding pictures. Coding all video pictures as P-frames also allows for reduction in the buffer size requirement.
In the latest H.264/AVC video coding standard, new features were introduced that enhanced error resiliency. The features included Arbitrary Slice Ordering (ASO) and Flexible Macroblock Ordering (FMO). A slice refers to some sub-unit of a picture having two or more macroblocks, often from a common row of macroblocks. Previously, slices had to be received in a particular order to be properly decoded. ASO allowed for decoding of slices received in arbitrary order. By combining Arbitrary Slice Order (ASO) with Flexible Macroblock Order (FMO), further enhancement of error resilience is provided.
From the viewpoint of the H.264/AVC specification, the intra refreshing technique provides certain benefits. With cyclic INTRA mode assignment, buffering delay is expected due to buffer size reduction. With cyclic INTRA mode assignment, I or IDR picture types might be removed, resulting in more uniform bit count distribution. Error resilience with or without ASO and FMO and error recovery is enhanced, resulting in better picture quality if errors occur.
However, there are certain disadvantages to IR. One disadvantage is that a certain amount of compression loss is expected. This is especially true for the case that the INTRA mode is improperly assigned, which can cause large compression loss particularly for those macroblocks with content that is complex but easily predicted. Furthermore, if IR is combined with ASO & FMO, a one frame delay is needed to extract the macroblock ordering information before starting decoding macroblock information. In addition, it is difficult to implement a random access point with intra refresh because there is no completely clean entry point, such as an I-frame, as is used for Instantaneous Decoder Refresh (IDR) in the H.264 codec.
From the viewpoint of video encoding/delivery applications, the IR technique can provide certain benefits. When IR is combined with ASO and FMO, an encoder can do a better job in region of interest (ROI) coding. The IR technique can also enhance error resilience when combined with ASO and FMO. The delivered visual quality is significantly improved if proper error recovery/concealment is enabled at the decoder side.
If only INTRA macroblocks are allowed (and I-frames are not allowed), the network jitter due to the more uniform picture types is diminished, and a “constant” bits per frame scenario is possible, the buffer size may be reduced significantly.
Intra Refresh does not allow for easy definition of a random access point within a coded video stream. A random access point is quite crucial for some video applications, such as error resiliency. Furthermore, certain VCR-like functionality (e.g., fast forward and fast rewind) cannot be implemented in an IR-coded video stream due to the lack of a clean entry point for these functions. To do fast forward or fast rewind with a coded video stream it is desirable for the coding of each macroblock in a given frame to be independent of the coding of other macroblocks in the same frame. One could implement fast forward or fast rewind without a random access point if the coded video stream included I-frames. However, the use of I-frames in the coded video stream is disadvantageous for the reasons noted above.
It is within this context that embodiments of the present invention arise.